"""

.. module:: resource
.. moduleauthor:: @author: 

*Description*:

The purpose of this module is ....

*Constants*:

.. data:: TRACE
        
    TRACE is the lowest logging level. All log messages are dumped.

.. seealso:: class :class:`datastore.database.DataStore`, :class:`datastore.uri.Uri`, :class:`datastore.spaces.Spaces`
"""

# Imports
#
import types
import xml.etree.ElementTree
import xml.dom.minidom
import xml.parsers.expat
import sys
import base64
import logging
import zlib

from datastore.uri import Uri


# Constants
#
#CRLF = "\r\n"
# extra logging level
TRACE = 5
    
# Classes
#

class Resource(object):
    """
    There are three parts to a resource: id, data and metadata. Metadata is a 
    set of keys (always a full Uri) and values. The type of the value must be 
    one of: Uri, None, Boolean, Int, Long, Float, Unicode or String. The 
    metadata key doesn't need to be unique. All values are returned if needed.

    The data is all non class private variables in the object (these 
    "self.__var" variables are stored as "__someclass_var" in the __dict__).

    The following types are recorded and faithfully recreated:
    * None, Boolean, Int, Long, Float, Unicode, Tuple, base64 and String.
    * Dict/ResourceDict (unordered) and List.
    * All other types are converted to string.

    When a new Resource is created it has no id. Content and metadata are added 
    as needed. Resource ID's are always returned as unicode, not int.

    When the Resource is put into the DataStore for the first time, the content
    is serialised and the metadata is fetched. The id is set once it has been
    allocated. A null id is used to detect that this is a new resource.

    On a subsequent put, the DataStore uses the existing id, serialised the
    content, and fetches the metadata.

    When a Resource is fetched from the DataStore, its id, content and metadata 
    are all loaded.

    Sample XML::

        <?xml version='1.0' encoding='UTF-8'?>
        <resource
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://yarra.googlecode.com/files/resource-1.0.xsd"
            id="001">
            <metadata>
                <item key="met">aaa</item>
                <item key="type">simple</item>
            </metadata>
            <content>
                <item key="junk" type="term:type/unicode">abc</item>
                <item key="b" type="term:type/boolean">False</item>
                <item key="morejunk" type="term:type/unicode">def</item>
                <item key="nill" type="term:type/none" />
                <list key="alist">
                    <item type="term:type/string">one</item>
                    <item type="term:type/string">two</item>
                    <item type="term:string">three</item>
                </list>
                <dict key="adict">
                    <item key="1" type="term:type/string">one</item>
                    <item key="2" type="term:type/string">two</item>
                    <list key="45">
                        <item type="term:type/string">four</item>
                        <item type="term:type/string">five</item>
                    </list>
                </dict>
            </content>
        </resource>
    
    If ResourceDict() is used instead of dict(), attribute access is available.
    For example res1.aaa.bbb is the same as resr1.aaa['bbb'] if aaa is a 
    ResourceDict().
    
    When a resource is deserialised, all dicts are created as ResourceDict().
    
    A Resource may be serialised and deserialised using the repr() function,
    where the format is python code can easily regenerate the resource using
    the eval() function.::
    
        res1 = Resource()
        res1.a = 10
        res1 == Resource(repr=repr(res1))
    
    The XML serialisation format and the repr format treat metadata defined
    with multiple=True differently when deserialising a serialised resource.::
    
        res.set_metadata(Uri('uuu'), 'abc', multiple=True)
    
    For the XML format the result is ['abc'] but for repr, the result is 'abc'.
    This only occurs for the XML format, when there is only one value present.
    
    A DeserialisationException will be raised if exec() fails for a new 
    Resource. This will most commonly be caused if an unsupported object type
    was in the resource that was serialised.
    
    WARNING: Any instance variable set here with a name that does NOT 
    start with "_" will be serialised!
    
    **Constants**:
    
    :param serialised: Serialised XML resource as a string
    :param repr: repr representation of the resource as a string
    :param type: set the resource type (default=None) string or Uri
    :param strict: boolean for exceptions for unknown types (default=False)
    :param empty: boolean True to create an empty resource for type checking
    :param warning: boolean, True means log validation warning.
    """
               
    def __init__(self, serialised=None, repr=None, **kwargs):
        # Constructor.
        #

        # private instance variables
        # see http://docs.python.org/3.1/tutorial/classes.html

        # need to process this first because we replace __dict__
        if repr is None:
            self._id = None
            # Each element is a dictionary with uri, value, and multiple flag. 
            # The key is a full uri.
            self._metadata = {} 
        else:
            try:
                d = eval(repr)
            except:
                raise DeserialisationException(
                        "Invalid repr() of a resource: %s" % repr)
            self.__dict__ = d['content']
            self._id = d['id']
            self._metadata = d['metadata']

        self._logger = logging.getLogger('yarra.datastore.resource')

        # kwargs used so we can use reserved names for keyword parameters.
        self._strict = kwargs.get('strict', False)
                    
        if serialised is not None:
            self.deserialise(serialised)
            
        type_value = kwargs.get('type', None)
        if type(type_value) == types.StringType:
            self.set_type(Uri(type_value))
        elif type(type_value) == type(Uri()):
            self.set_type(type_value)

        # An empty resource is not valid unless explicitly requested.
        if not kwargs.get('empty', False):
            if (serialised is None) and (repr is None) and (type_value is None):
                raise ValueError(
                "Resource must specify either a serialised resource or a type")  

        # An empty resource is not value unless explicitly requested.
        self._warnings = kwargs.get('warnings', True)

    def get_id(self):
        """
        This function returns the resource id. For example::
        
            id = res.id
        
        :rtype: String identifier used by datastore.
        """
        
        return self._id

    def set_id(self, value):
        """
        Only set by the DataStore. For example::
        
            res.id = <identifier>
        
        :param value: string identifier used by datastore.
        :rtype: None
        """
        
        if value is None:
            raise TypeError("Resource ID cannot be None.")
        else:
            self._id = str(value)

    def get_metadata(self, key, default=None):
        """
        This function returns a metadata value (of any supported  
        resource variable type, including None) for the specified Uri 
        key or an exception if the key doesn't exist.
        
        If the metadata was set with the multiple option, a sorted list 
        of values is always returned.
        
        This function returns a sorted list containing one or more 
        values.
        
        :param key: Uri key.
        :param default: A default value if the metadata element is not found.
        :rtype: any supported resource variable type single, or a list for 
            multiple values. 
        """
        
        if type(key) != type(Uri()):
            raise TypeError("Key (%s) must be a Uri." % str(key))
        full_uri = key.full()
        #
        if full_uri in self._metadata:
            # either a value or a list (if multiple values)
            return self._metadata[full_uri]['value']
        else:
            if default is None:
                raise IndexError("Metadata key (%s) not found." % str(key))
            else:
                return default

    def set_metadata(self, key, value, multiple=False):
        """
        This function adds a new key/value metadata entry.
        
        If mutliple is set then there may be multiple values for the key.
        It must be set the first time that it is needed.
        
        :param key: Uri
        :param value: None or one of the valid types for metadata.
        """
        
        if type(key) != type(Uri()):
            raise TypeError("Key (%s) must be a Uri." % str(key))
        full_uri = key.full()
        #
        if full_uri in self._metadata:
            # update or append to an existing entry
            if self._metadata[full_uri]['multiple']:
                # append a new element to multiple value entry.
                self._metadata[full_uri]['value'].append(value)
            else:
                # existing single value entry
                if multiple:
                    raise TypeError(
                        "Cannot convert a single entry (%s) to multiple." % \
                        str(key))
                else:
                    # update the single entry.
                    self._metadata[full_uri]['value'] = value
        else:
            # create a new entry
            if multiple:
                if full_uri in self._metadata:
                    # multiple already exists, just append
                    self._metadata[full_uri]['value'].append(value)
                else:
                    # new multiple needed
                    meta = {}
                    meta['uri'] = key
                    meta['value'] = [value]
                    meta['multiple'] = True
                    self._metadata[full_uri] = meta
            else:
                # new single needed
                meta = {}
                meta['uri'] = key
                meta['value'] = value
                meta['multiple'] = False
                self._metadata[full_uri] = meta
                    
    def del_metadata(self, key, value=None):
        """
        This function deletes a metadata entry. This will raise 
        an exception if the key or value are not found. If this was the 
        last value for the key, the key is removed.
        
        If value is None, a single entry is deleted, or for multiple, all
        entries are deleted.  
        
        :param key: Uri (not needed for single value delete)
        :param value: None or one of the valid types for metadata.
        """
        
        if type(key) != type(Uri()):
            raise TypeError("Key (%s) must be a Uri." % str(key))
        full_uri = key.full()
        #
        if full_uri in self._metadata:
            meta = self._metadata[full_uri]
            if meta['multiple']:
                if value is None:
                    del self._metadata[full_uri]
                else:
                    try:
                        meta['value'].remove(value)
                    except ValueError:
                        raise IndexError(
                            "Metadata value (%s) not found for key (%s)." % \
                            (str(value), str(key)))
                    if len(meta['value']) == 0:
                        del self._metadata[full_uri]
            else:
                del self._metadata[full_uri]
        else:
            raise IndexError("Metadata key (%s) not found." % str(key))

    def metadata_has_key(self, key):
        """
        This function returns True if the argument key exists, or False 
        otherwise.
        
        :param key: Uri
        :rtype: Boolean - True if metadata has argument key.
        """
        
        if type(key) != type(Uri()):
            raise TypeError("Key (%s) must be a Uri." % str(key))
        #
        return key.full() in self._metadata

    def get_metadata_keys(self):
        """
        This function returns a sorted list of all of the unique metadata 
        keys, or none if there are no keys. ['uri'] is used rather than the
        full uri stored as the dictionary key.
        
        :rtype: Sorted list of Uri
        """
        
        keys = []
        for key in self._metadata.keys():
            keys.append(self._metadata[key]['uri'])
        return sorted(keys)

    def get_type(self):
        """
        This function returns the resource type or None if there is no type.
        
        :rtype: Uri or None
        """
        
        if not self.metadata_has_key(Uri('term:base/datastore/resource/type')):
            return None
        uri = self.get_metadata(Uri('term:base/datastore/resource/type'))
        if type(uri) == type(Uri()):
            return uri
        else:
            raise TypeError("Resource type (%s) must be a Uri." % str(uri))
     
    def set_type(self, uri):
        """
        This function replaces the resource type.
        
        :param value: Uri representing the Resource type.
        :rtype: None.
        """
        
        if type(uri) == type(Uri()):
            self.set_metadata(Uri('term:base/datastore/resource/type'), uri)
        else:
            raise TypeError("Value (%s) must be a Uri." % str(uri))

    def dc_title(self, value=None, default=None):
        doc = "Get or set the dc:title metadata element."
        """
        This function returns the metadata value.
        :rtype: String
        """
        
        if value is None:
            return self.get_metadata(Uri('dc:title'), 
                                     default=default).strip()
        else:
            self.set_metadata(Uri('dc:title'), value)
            return value

    def dc_description(self, value=None, default=None):
        doc = "Get or set the dc:description metadata element."
        """
        This function returns the metadata value.
        :rtype: String
        """
        
        if value is None:
            return self.get_metadata(Uri('dc:description'), 
                                     default=default).strip()
        else:
            self.set_metadata(Uri('dc:description'), value)
            return value
 
    def get_var(self, name, default=None):
        """
        This function returns the variables of the current instance of
        Resource that has the argument name. This gives support for variables
        that don't have a valid python variable name.
        
        :param name: Variable name.
        :rtype: variable value
        """
        
        name_str = str(name)
        if name_str.startswith('_'):
            raise IndexError("Cannot return private variable (%s)." % name_str)
        else:
            if name_str in self.__dict__:
                return self.__dict__[name_str]
            else:
                if default is None:
                    raise IndexError("Unknown variable (%s)." % name_str)
                else:
                    return default
  
    def set_var(self, name, value):
        """
        This function sets a variable in the current instance of Resource. This
        gives support for variables that don't have a valid python variable 
        name.
        
        :param name: Variable name as string
        :param value: Any valid Python value (but not all can be serialised).
        """
        
        name_str = str(name)
        if name_str.startswith('_'):
            raise IndexError("Cannot set private variable (%s)." % name_str)
        else:
            self.__dict__[name_str] = value

    def del_var(self, name):
        """
        This function deletes a variable in the current instance of Resource. 
        This gives support for variables that don't have a valid python 
        variable name.
        
        :param name: Variable name
        """
        
        name_str = str(name)
        if name_str.startswith('_'):
            raise IndexError("Cannot delete private variable (%s)." % name_str)
        else:
            del self.__dict__[name_str]

    def has_var(self, name):
        """
        This function returns True if the argument variable name 'key' exists
        in the current instance of Resource. This gives support for variables 
        that don't have a valid python variable name.
        
        :param key: Variable name
        :rtype: Boolean
        """
        
        name_str = str(name)
        if name_str.startswith('_'):
            raise IndexError("Cannot check private variable (%s)." % name_str)
        else:        
            return str(name_str) in self.__dict__

    def get_vars(self):
        """
        This function returns those variables on the current instance of
        Resource that have names that don't begin with '_'. ie. omitting
        variables of the form "_someclass__var".

        :rtype: ResourceDict
        """
        
        vars = ResourceDict()
        for var in self.__dict__:
            if not str(var).startswith('_'):
                vars[var] = self.__dict__[var]
        return vars

    def get_var_part(self, part):
        """
        This function returns some part of the resource that has been
        specified as a string.
        
        If part is None or an empty string, the contents of the resource
        are returned.
        
        If part, or part of part, does not exist, then return None.
        
        :param key: Variable part.
        :rtype: variable (either simple or complex)
        """
        
        # Get the resources variables as a dict to start the process.
        current = self.get_vars()
        if (part is not None) and (len(str(part)) > 0):
            try:
                parts = part.split(".")
            except:
                details = "Part '%s' is not a string." % (part)
                raise GetPartException(details)
            try:
                for p in parts:
                    #print("Part: %s" % p)
                    if type(current) in [types.DictType, type(ResourceDict())]: 
                        # The dict key might be a string or an int. Try both.
                        # Other number types are not supported.
                        if p in current:
                            current = current[p]
                        else:
                            try:
                                if int(p) in current:
                                    current = current[int(p)]
                                else:
                                    # neither str(p) nor int(p) in 'current', 
                                    # so return None.
                                    return None
                            except:
                                # str(p) not in 'current' and p not an number, 
                                # so return None.
                                return None
                    else:
                        # Assume its a list-like type.
                        current = current[int(p)]
            except:
                raise GetPartException(
                            "Unable to get '%s' for the Resource." % (part))
        return current

    def validate(self, spaces, type=None, warning=True):
        """
        This method will validate this resource against definition resources in
        spaces. If all validation tests are past, True is returned, otherwise
        False is returned.
        
        #. If specified, verify that the resource is of the correct type.
        
        #. For each metadata element, a metadata definition resource must
           exist in spaces. A warning is logged if the definition resource is 
           not found.
        
        #. If the metadata definition contains a type variable, then the
           metadata value type must match the type uri.
        
        #. If the metadata definition contains a cardinality variable, then the
           metadata value must match the cardinality.

        #. A definition resource must exist for the resource's type Uri. A 
           warning is logged if the resource is not found.
        
        #. Each metadata element in the resource must exist in the resource
           definition.
        
        #. Each content variable in the resource must exist in the resource
           definition.
        
        Warnings are logged if the constructor warning flag is set and the
        flag on this method. Warning log entries can be disabled for resources
        created very early in node start-up (i.e. definition resources).
        
        :param spaces: local of all definition resources.
        :param warning: if True, warnings are logged.
        :rtype: Boolean.
        """
        
        if self.dc_title(default='') is not '':
            self._logger.log(TRACE, "Validating resource (%s)", self.dc_title())
        if not self._warnings:
            warning = False
        val = True
        # (0) Check the type of this resource.
        if type is not None:
            if Uri(type) != self.get_type():
                if warning:
                    self._logger.log(logging.WARNING,
                        "Resource (%s) type is not (%s).",
                        str(self.get_type()), type)
                val = False
        # (1) Metadata definition must exist for each metadata element.
        for meta_uri in self.get_metadata_keys():
            if spaces.exists(meta_uri):
                meta = spaces.get(meta_uri)
                # (2) If the definition specifies a type, then validate it.
                if meta.has_var('type'):
                    meta_type = self.check_type(self.get_metadata(meta_uri))
                    if Uri(meta_type) != meta.get_var('type'):
                        if warning:
                            self._logger.log(logging.WARNING,
                                "Metadata resource (%s) type is not (%s).",
                                str(meta_uri), meta_type)
                        val = False
                # (3) If the definition specifies carinality, then validate it.
                if meta.has_var('multiple'):
                    multiple = self._metadata[meta_uri]['multiple']
                    if meta['multiple'] == multiple:
                        if warning:
                            self._logger.log(logging.WARNING, "Metadata " + \
                                "resource (%s) type multiple is not (%s).",
                                str(meta_uri), multiple)
                        val = False
            else:
                if warning:
                    self._logger.log(logging.WARNING,
                        "Metadata resource (%s) was not found in spaces.",
                        str(meta_uri))
                val = False
        # (4a) A resource type must be specified.
        if not self.metadata_has_key(Uri('term:base/datastore/resource/type')):
            if warning:
                self._logger.log(logging.WARNING,
                    "Resource type not found.")
            return False
        # (4b) Definition resource must exist
        if not spaces.exists(self.get_type()):
            if warning:
                self._logger.log(logging.WARNING,
                    "Definition resource (%s) was not found in spaces.",
                    str(self.get_type()))
            return False
        # (5) Check that each metadata element is in the resource definition.
        def_res = spaces.get(self.get_type())
        if def_res.has_var('metadata'):
            for meta_uri in self.get_metadata_keys():
                # type metadata element not explicitly needed in each definition
                # metadata added during database build, won't be in the def.
                if meta_uri in [Uri('term:base/datastore/resource/type'),
                                Uri('term:base/build/autoload_suffix'),
                                Uri('term:base/build/revision')]:
                    continue
                if str(meta_uri) in def_res.metadata.keys():
                    pass
                else:
                    if warning:
                        self._logger.log(logging.WARNING,
                            "Resource metadata (%s) not found in definition.",
                            str(meta_uri))
                    val = False
        # (6) Check that each content variable is in the resource definition.
        if def_res.has_var('content'):
            for var in self.get_vars():
                if var in def_res.content.keys():
                    pass
                else:
                    if warning:
                        self._logger.log(logging.WARNING, "Resource content" + \
                             " variable (%s) not found in definition. %s", var, 
                             def_res.dc_title())
                    val = False
        # Tests done.
        return val
    
    def serialise(self):
        """
        This function returns the XML representation of the current Resource
        instance.
        
        Metadata and dicts are sorted, so a comparison of the serialised
        resource can be used to test equality.
        
        Although I would prefer to use <int>123</int>, using a generic item
        tag and a type attribute makes adding new types transparent.
        
        :rtype: String
        """

        root = self._gen_xml_header()
        
        if self._id is not None:
            root.attrib['id'] = self._id
        if len(self._metadata) != 0:
            meta = xml.etree.ElementTree.SubElement(root, 'metadata')
            for key in self.get_metadata_keys():
                value_list = self.get_metadata(key)
                if type(value_list) != types.ListType:
                    value_list = [value_list]
                for value in sorted(value_list):
                    # Duplicate keys and/or values are permitted.
                    meta_element = xml.etree.ElementTree.SubElement(meta,'item')
                    meta_element.attrib['key'] = str(key.shorten())
                    # A value of none is permitted.
                    self._type_and_add(value, meta_element)        
        content = xml.etree.ElementTree.SubElement(root, 'content')
        self._gen_xml_dict(content, self.get_vars(), None)
        return xml.etree.ElementTree.tostring(root, 'UTF-8')

    def serialise_content(self): 
        """
        This function returns the XML representation of the current Resource
        instance's content field.
        
        The resource ID and metadata are omitted. This method is used by the
        data store.
        
        :rtype: String
        """
        
        root = self._gen_xml_header()
        content = xml.etree.ElementTree.SubElement(root, 'content')
        self._gen_xml_dict(content, self.get_vars(), None)
        return xml.etree.ElementTree.tostring(root, 'UTF-8')

    def _gen_xml_header(self):
        """
        This function returns a XML ElementTree object representing the
        Resource header.
        
        :rtype: ElementTree
        """
        
        root = xml.etree.ElementTree.Element('resource')
        root.attrib['xmlns:xsi'] = "http://www.w3.org/2001/XMLSchema-instance"
        root.attrib['xsi:schemaLocation'] = \
            "http://yarra.googlecode.com/files/resource-1.0.xsd"
        return root
    
    def _gen_xml(self, et, bit, tag, key=None):
        """
        See http://docs.python.org/library/types.html#module-types
        
        :param et: <class> or <type>
        :param bit: <class> or <type>
        :param tag: <class> or <type>
        :param key: <class> or <type>
        """
        
        if type(bit) in [types.DictType, type(ResourceDict())]:
            self._gen_xml_dict(et, bit, 'dict', key)
        elif type(bit) == types.ListType:
            self._gen_xml_list_tuple(et, bit, 'list', key)
        elif type(bit) == types.TupleType:
            self._gen_xml_list_tuple(et, bit, 'tuple', key)
        else:
            d = xml.etree.ElementTree.SubElement(et, tag)
            self._type_and_add(bit, d, key)
            if key is not None:
                d.attrib['key'] = str(key)

    def _type_and_add(self, bit, d, key=None):
        """
        This function adds a 'type' attribute to the argument 
        ElementTree.SubElement 'd', with a value that is a local Uri 
        representing the type of the argument 'bit'.
        
        :param bit: <class> or <type>
        :param d: <class> or <type>
        :param key: <class> or <type>
        """
        
        t = self.check_type(bit)
        if t is None:
            if self._strict:
                raise DeserialisationException(
                    "Unknown type for item: %s." % str(key))
            else:
                self._logger.log(logging.WARNING, 
                    "Unknown type for item '%s'. Setting to string.", str(key))
            d.text = str(bit)
            d.attrib['type'] = 'term:type/string'
        elif t == 'term:type/none':
            d.attrib['type'] = t
        elif t == 'term:type/string':
            d.attrib['type'] = t
            d.text = str(bit)
        elif t == 'term:type/base64':
            d.attrib['type'] = t
            d.text = bit.encode()
        else:
            d.attrib['type'] = t
            d.text = unicode(bit)

    def check_type(self, value):
        """
        Return the type string for a value.
        For serialise, dict, list and tuple have been previously processed.
        
        :param value:
        :rtype: String or None
        """
        
        if   type(value) == types.NoneType:         return 'term:type/none'
        elif type(value) == types.BooleanType:      return 'term:type/boolean'
        elif type(value) == types.IntType:          return 'term:type/int'
        elif type(value) == types.LongType:         return 'term:type/long'
        elif type(value) == types.FloatType:        return 'term:type/float'
        elif type(value) == types.UnicodeType:      return 'term:type/unicode'
        elif type(value) == types.StringType:       return 'term:type/string'
        elif value.__class__.__name__ == 'Data':    return 'term:type/base64'
        elif value.__class__.__name__ == 'Uri':     return 'term:type/uri'
        elif type(value) in [types.DictType, type(ResourceDict())]:
                                                    return 'term:type/dict'
        elif type(value) == types.ListType:         return 'term:type/list'
        elif type(value) == types.TupleType:        return 'term:type/tuple'
        else:                                       return None
           
    def _gen_xml_dict(self, e, bit, tag, key=None):
        """
        This function ... 
        
        :param e: <class> or <type>
        :param bit: <class> or <type>
        :param tag: <class> or <type>
        :param key: <class> or <type>
        """
        
        if tag is None:
            d = e
        else:
            d = xml.etree.ElementTree.SubElement(e, tag)
            if key is not None:
                d.attrib['key'] = str(key)
        bit_keys = bit.keys()
        bit_keys.sort()
        for b in bit_keys:
            self._gen_xml(d, bit[b], 'item', b)

    def _gen_xml_list_tuple(self, e, bit, tag, key=None):
        """
        This function ... 
        
        :param e: <class> or <type>
        :param bit: <class> or <type>
        :param tag: <class> or <type>
        :param key: <class> or <type>
        """
        
        d = xml.etree.ElementTree.SubElement(e, tag)
        if key is not None:
            d.attrib['key'] = str(key)
        for b in bit:
            self._gen_xml(d, b, 'item')

    def deserialise(self, content):
        """
        This function de-serialises the XML 'content'. 
        
        :param content: <class> or <type>
        """
        
        doc = self._parse_xml(content)
        self._parse_resource(doc)
        # process metadata and the actual resource content
        for part in doc.childNodes:
            if part.nodeType == part.TEXT_NODE:
                pass
            elif part.nodeType == part.COMMENT_NODE:
                pass
            elif part.tagName == 'metadata': 
                self._parse_metadata(part)
            elif part.tagName == 'content': 
                self._parse_content(part)
            else:
                #FIXME: No test case
                raise DeserialisationException(
                    "Unknown element under resource: %s." % part.tagName)

    def _parse_xml(self, content):
        """
        This function parses the argument 'content' XML to produce a
        DocumentElement instance.
        
        :param content: String - XML to parse.
        :rtype: DocumentElement
        """
        
        try:
            dom = xml.dom.minidom.parseString(content)
        except TypeError:
            raise DeserialisationException("Expected string or buffer.")
        except xml.parsers.expat.ExpatError:
            raise DeserialisationException(
                                "Expat parse error: %s" % sys.exc_info()[1])
        return dom.documentElement
       
    def _parse_resource(self, res):
        """
        This function parses the argument Resource 'res' and copies the 
        Resource's 'id' attribute in the current Resource instance.
        
        :param res: Resource - the Resource to parse for 'id'.
        """
        
        if res.tagName != 'resource':
            raise DeserialisationException("Missing 'resource' root element.")
        if res.getAttributeNode('id') is not None:
            self._id = res.getAttribute('id')

    def _parse_metadata(self, metadata):
        """
        This function parses the argument DOM Node 'metadata' and copies all
        the contents of all 'key' fields to metadata in the current 
        Resource instance. 
        
        Any existing metadata is discarded.
        
        :param metadata: Node
        """
        
        self._metadata = {}
        if metadata.hasChildNodes(): 
            for item in metadata.childNodes:
                if item.nodeType == item.TEXT_NODE:
                    continue
                if item.nodeType == item.COMMENT_NODE:
                    continue
                if item.getAttributeNode('key') is not None:
                    # self._parse_text(item.childNodes) might return None.
                    uri = Uri(item.getAttribute('key'))
                    if self.metadata_has_key(uri):
                        value = self.get_metadata(uri)
                        if type(value) == types.ListType:
                            # already multiple entry type, so just add another.
                            self.set_metadata(uri, 
                                self._deserialise_content(item))
                        else:
                            # convert a single to multiple if we find another.
                            self.del_metadata(uri, value)
                            self.set_metadata(uri, value, multiple=True)
                            self.set_metadata(uri, 
                                self._deserialise_content(item))
                    else:
                        self.set_metadata(uri, self._deserialise_content(item))
                else:
                    raise DeserialisationException(
                                    "Missing key for metadata element.")                    

    def _parse_content(self, content):
        """
        This function ... 
        
        :param param_1: <class> or <type>
        :param param_2: <class> or <type>
        :rtype: <return type>
        """
        
        if content.hasChildNodes():
            self.__dict__.update(self._deserialise_content(content))

    def _deserialise_content(self, node):
        """
        This function processes the content section of a Resource or
        a metadata value. 
        
        :param node: <class> or <type>
        :rtype: One of None, ResourceDict, List, tuple, Boolean, int, 
            long, float, unicode, Uri or string.
        """
        
        if (node.tagName == 'dict') or (node.tagName == 'content'):
            new_dict = ResourceDict()
            for item in node.childNodes:
                if item.nodeType == item.TEXT_NODE:
                    continue
                if item.nodeType == item.COMMENT_NODE:
                    continue
                if item.getAttributeNode('key') is None:
                    raise DeserialisationException(
                            "Missing key for element: %s." % item.tagName)
                item_key = item.getAttribute('key')
                new_dict[str(item_key)] = self._deserialise_content(item)
            return new_dict
        elif node.tagName == 'list':
            new_list = []
            for item in node.childNodes:
                if item.nodeType == item.TEXT_NODE:
                    continue
                if item.nodeType == item.COMMENT_NODE:
                    continue
                new_list.append(self._deserialise_content(item))
            return new_list
        elif node.tagName == 'tuple':
            new_tuple = ()
            for item in node.childNodes:
                if item.nodeType == item.TEXT_NODE:
                    continue
                if item.nodeType == item.COMMENT_NODE:
                    continue
                new_tuple += self._deserialise_content(item),
            return new_tuple
        elif node.tagName == 'item':
            # Only items in a dict or content will have a key.
            item_key = None
            if node.getAttributeNode('key') is not None:
                item_key = node.getAttribute('key')
                
            # every item must have a type.    
            if node.getAttributeNode('type') is None:
                raise DeserialisationException(
                            "Missing type for element: %s." % node.tagName)
            item_type = Uri(node.getAttribute('type'))
            
            # 'none' type's value is ignored.
            if item_type == Uri('term:type/none'):
                return None
            item_text = self._parse_text(node.childNodes)
            
            # All items must have a value except string, which should result in
            # an empty string.
            #
            if item_text is None:
                if item_type in \
                    [Uri('term:type/string'), Uri('term:type/base64')]:
                    item_text = ""
                else:
                    raise DeserialisationException(
                                "Element text is None for: %s (%s)." % \
                                (item_key, item_type))
            
            return self._item_type(item_type, item_text, node.tagName)
        else:
            raise DeserialisationException(
                            "Unknown element(%s)." % node.tagName)

    def _item_type(self, item_type, item_text, tag_name):
        """
        This function returns the argument 'item_text' but cast to the 
        internal (python) type (or Uri) that is represented by the argument
        'item_type'.
        
        :param term: <class> or <type>
        :param item_type: <class> or <type>
        :param item_text: <class> or <type>
        :param tag_name: <class> or <type>
        :rtype: One of Boolean, int, long, float, unicode, Uri or string.
        """
        
        if item_type == Uri('term:type/boolean'):
            if item_text.strip().lower() == 'true':
                return True
            elif item_text.strip().lower() == 'false':
                return False
            else:
                raise DeserialisationException(
                        "Invalid boolean value for element(%s)." % tag_name)
        elif item_type == Uri('term:type/int'):
            try:
                return int(item_text)
            except:
                raise DeserialisationException(
                        "Invalid int value for element(%s)." % tag_name)
        elif item_type == Uri('term:type/long'):
            try:
                return long(item_text)
            except:
                raise DeserialisationException(
                        "Invalid long value for element(%s)." % tag_name)
        elif item_type == Uri('term:type/float'):
            try:
                return float(item_text)
            except:
                raise DeserialisationException(
                        "Invalid float value for element(%s)." % tag_name)
        elif item_type == Uri('term:type/unicode'):
            # XML decoding errors are detected at parse time.
            # No need to check again here.
            return unicode(item_text)
        elif item_type ==  Uri('term:type/base64'):
            return Data(encoded=item_text)
        elif item_type == Uri('term:type/uri'):
            return Uri(item_text)
        else:
            # this includes type string and uri
            return str(item_text)        
        
    def _parse_text(self, nodelist):
        """
        This function returns an string that is the concatenation of all
        XML TEXT nodes in the argument 'nodelist'. 
        
        Fetch the text children for a node. Ignore other nodes.
        Carefully distinguish between an empty text node, returned as an empty
        string and no text nodes, returned as None.

        These are the elements we might find here:
            Node.ELEMENT_NODE        - ignore embedded valid tags such as <br />
            Node.TEXT_NODE           - add to text
            Node.CDATA_SECTION_NODE  - add to text
            Node.ENTITY_REFERENCE_NODE            - ignore
            Node.PROCESSING_INSTRUCTION_NODE      - ignore
            Node.COMMENT_NODE                     - ignore
            Node.NOTATION_NODE                    - ignore
        
        :param nodelist: List - list of Nodes.
        :rtype: String or None
        """
        
        rc = None
        for node in nodelist:
            if node.nodeType == node.TEXT_NODE:
                if rc is None:
                    rc = node.data
                else:
                    rc += node.data
            elif node.nodeType == node.CDATA_SECTION_NODE:
                if rc is None:
                    rc = node.data
                else:
                    rc += node.data
            #else:
            #    print(node.nodeType, repr(node))
        return rc

    def copy(self):
        """
        Returns a deep copy of this resource. That is, the resource instance and
        all data within it is a copy of the original.
        
        This transformation to and from XML is not that efficient, especially
        because spaces makes a copy for get-like and put-like operations. It
        is however, robust.
        """
        return Resource(repr=repr(self))
    
    def __cmp__(self, other):
        """
        This function compares the current instance with the argument Resource
        instance 'other'. This should return a negative integer if self < other, 
        zero if self == other, a positive integer if self > other.
        
        This function will be called by Python for Resource comparisons, to sort
        the Resources by 'id'.
        
        :param other: Resource - other Resource to compare against current 
            instance.
        :rtype: int
        """
        
        if other is None:
            # if other is None, less just say that self is greater than other
            return 1
        elif str(self) == str(other):
            return 0
        else:
            if type(other) is type(Resource(empty=True)):
                if self.get_id() < other.get_id():
                    return -1
                else:
                    return 1
            else:
                return -1
    
    def __str__(self):
        """
        This function returns a string representation of the current instance
        of Resource.
        
        :rtype: String
        """
        
        return self.serialise()
    
    def __repr__(self):
        """
        This function returns a repr representation of the current instance
        of Resource, suitable for exec'ing back to a resource.
        
        :rtype: String
        """
        
        return repr({'id': self._id, 
                     'metadata': self._metadata, 
                     'content': self.get_vars()})
    
    def load_from_file(self, filename):
        """
        This function loads the current instance of Resource with the contents
        of the argument 'filename'. This function primarily supports testing, 
        especially loading uri definitions.
        
        :param filename: String
        """
        
        res_file = open(filename, 'r')
        res_xml = Resource(res_file.read())
        res_file.close()
        res_file = None
        self.deserialise(str(res_xml))
    
    def pretty_print(self):
        """
        This function displays the Resource in a pretty manner.
        """
        
        parser = xml.etree.ElementTree.XMLTreeBuilder(
                                        target=ResourcePrettyPrint())
        parser.feed(self.serialise())
        return parser.close()
            

class ResourceException(Exception):
    pass

class DeserialisationException(ResourceException):
    pass

class GetPartException(ResourceException):
    pass


class ResourceDict(dict):
    """
    This dict allows attribute style access to the dictionary keys. Obviously, 
    the keys must be valid attributes names in order to use these methods. All 
    normal exceptions are raised (except not found, which returns None).
    """
    
    def __getattr__(self, name):
        """
        :param name:
        :rtype: variable content or None if not found.
        """
        
        return self.__getitem__(name)
    
    def __setattr__(self, name, value):
        """
        :param name:
        :param value:
        """
        
        return self.__setitem__(name, value)
    
    def __repr__(self):
        """
        """
        
        return "ResourceDict(%s)" % dict(self)


class Data(object):
    """
    Allow binary data to be serialised as base64. Data allows an instance to
    be created using binary data. When a serialised Resource is re-created,
    a Data instance may be recreated using already encoded data (as used for
    repr).
    
    **Constants**:
    
    :param raw: binary data to be used to construct Data()
    :param encoded: Create Data() using already encoded data.
    """
    
    def __init__(self, raw=None, encoded=None):
        self.data = None
        if raw is not None:
            self.data = base64.b64encode(zlib.compress(raw))
        if encoded is not None:
            if len(encoded) == 0:
                self.data = None
            else:
                self.data = encoded
    
    def encode(self):
        return self.data
    
    def decode(self):
        if self.data is None:
            return None
        else:
            return zlib.decompress(base64.b64decode(self.data))
    
    def __str__(self):
        if self.data is None:
            return ''
        else:
            return self.decode()
        
    def __repr__(self):
        if self.data is None:
            return "Data()"
        else:
            return "Data(encoded='%s')" % self.data

    def __cmp__(self, other):
        return cmp(self.data, other.encode())


class ResourcePrettyPrint:
    """
    This class prints the XML representation of a Resource in a pretty manner.
    
    **Constants**:
    """
    
    # Class constants
    #
    
    def __init__(self):
        self.depth = 0
        self.str = ""
    
    def start(self, tag, attrib):
        """
        This function is called for each opening tag.
        
        :param tag: String containing HTML tag name.
        :param attrib: Dictionary containing HTML tag attributes.        
        """    
        
        self.indent()
        
        if (len(attrib) > 0):
            self.str += "<%s" % tag
            for key in attrib.keys():
                self.str += ' %s' % key
                self.str += '="%s"' % attrib[key]
            self.str += ">"
        else:
            self.str += "<%s>" % tag
        self.str += "\n"
        
        self.depth += 1
    
    def end(self, tag):
        """
        This function is called for each closing tag.
        
        :param tag: String containing HTML tag name.
        """
        
        self.depth -= 1
        
        self.indent()
        self.str += "</%s>" % tag
        self.str += "\n"
    
    def data(self, data):
        """
        This function displays the data.
        
        :param data: String containing HTML tag data.
        """
        
        self.indent()
        self.str += data.strip()
        self.str += "\n"
    
    def close(self):
        """
        This function is called when all of the data has been parsed.
        """
        
        return self.str

    def indent(self):
        for i in range(0, self.depth):
            self.str += "  "